Even with the introduction of fabrication technology advances, specialized architectures, and the application of optimization techniques, the trend is disturbing. Couple that with the explosion in edge devices that are adding increasing amounts of intelligence and it becomes clear that something dramatic has to happen. Today, most of the efforts are related to physically bringing the memory closer to the compute and where possible putting enough inside the package such the I/O costs are reduced. "One of the foundational ideas of analog is you can actually compute in the memory cell itself," says Tim Vehling, senior vice president for product and business development at Mythic. "If you look around the house, look at how many items are actually plugged into the wall in standby mode, all taking 5 or 10 watts," says Alexander Wakefield, scientist at Synopsys.

The power consumed by machine learning is exploding, and while advances are being made in reducing the power consumed by them, model sizes and training sets are increasing even faster. Even with the introduction of fabrication technology advances, specialized architectures, and the application of optimization techniques, the trend is disturbing. Couple that with the explosion in edge devices that are adding increasing amounts of intelligence and it becomes clear that something dramatic has to happen. The right answer is not to increase the world's energy production. It is to use what we have more wisely. The industry has to start taking total energy consumed by a machine learning application seriously, and that must include asking the question, 'Is the result worth the power expenditure?'

The past two decades has seen machine learning (ML) transformed from an academic curiosity to a multi-billion dollar industry, and a centerpiece of our economic, social, scientific, and security infrastructure. Much work in machine learning has drawn on research in optimization, motivated by large-scale applications requiring analysis of massive high-dimensional data. In this talk, I'll argue that the growing importance of networked data environments, from the Internet to cloud computing, requires a fundamental rethinking of our basic analytic tools. My thesis will be that ML needs to shift from its current focus on optimization to equilibration, from modeling the world as uncertain, but stationary and benign, to one where the world is non-stationary, competitive, and potentially malicious. Adapting to this new world will require developing new ML frameworks and algorithms.